Skip to content

ci: daily LoC report to Slack (daily) and Telegram (weekly)#368

Open
MegaRedHand wants to merge 4 commits into
mainfrom
feat/daily-loc-report
Open

ci: daily LoC report to Slack (daily) and Telegram (weekly)#368
MegaRedHand wants to merge 4 commits into
mainfrom
feat/daily-loc-report

Conversation

@MegaRedHand
Copy link
Copy Markdown
Collaborator

Summary

  • Adds .github/workflows/daily_loc_report.yml: scheduled at 0 0 * * * UTC plus workflow_dispatch (with target=test/prod and a post_telegram opt-in for manual runs).
  • Counts Rust LoC per workspace member via tokei + cargo metadata, with day-over-day deltas computed against the previous run's report (restored from the branch-scoped Actions cache).
  • Publishes a Slack Block Kit message every day, and an HTML-formatted Telegram message on Mondays UTC (or any manual run that opts in).

Slack and step-summary outputs include total Rust LoC and a per-crate breakdown sorted descending. Excludes tests/, benches/, examples/.

Required secrets / variables

Secret Used for
ETHLAMBDA_GENERAL_SLACK_WEBHOOK Prod Slack channel (scheduled + manual target=prod)
ETHLAMBDA_TEST_SLACK_WEBHOOK Test Slack channel (manual target=test)
TELEGRAM_BOT_TOKEN Telegram bot auth
TELEGRAM_ETHLAMBDA_CHAT_ID Prod Telegram chat (scheduled Mondays + manual target=prod)
TELEGRAM_ETHLAMBDA_TEST_CHAT_ID Test Telegram chat (manual target=test)

Secret names mirror ethrex's ETHREX_* pattern — adjust if there are existing conventions in the org.

Test plan

  • Configure the secrets above in the repo settings.
  • Run the workflow manually with target=test and post_telegram=false — verify the Slack test channel receives the daily message and the workflow step summary shows the per-crate table.
  • Run again with target=test and post_telegram=true — verify Telegram test chat receives the weekly-style message.
  • Confirm subsequent runs on the same branch show day-over-day deltas (e.g. (+12) / (-3)).
  • Optionally trigger one target=prod manual run after secrets are wired to validate the production channel/chat before the first scheduled fire.

Adds a scheduled workflow that counts Rust LoC in the workspace and
publishes a report to Slack daily and to Telegram weekly (Monday UTC).

Day-over-day deltas are computed against the previous run's report,
cached on the branch under `loc-report-<ref_name>-` keys.

Required secrets:
  ETHLAMBDA_GENERAL_SLACK_WEBHOOK   prod Slack channel
  ETHLAMBDA_TEST_SLACK_WEBHOOK      test Slack channel (manual runs)
  TELEGRAM_BOT_TOKEN                bot token
  TELEGRAM_ETHLAMBDA_CHAT_ID        prod Telegram chat
  TELEGRAM_ETHLAMBDA_TEST_CHAT_ID   test Telegram chat (manual runs)
@github-actions
Copy link
Copy Markdown

🤖 Kimi Code Review

Critical Issues

  1. Non-existent Action Version (.github/workflows/daily_loc_report.yml:35)
    actions/checkout@v6 does not exist (latest stable is v4). This will cause the workflow to fail immediately.

    # Change to:
    uses: actions/checkout@v4
  2. Unescaped HTML in Telegram Output (.github/scripts/generate_loc_report.sh:169-170)
    The Telegram script uses parse_mode=HTML, but crate paths are interpolated without escaping. If a path contains &, <, or >, it will break the message format or potentially allow HTML injection (low risk for internal CI, but incorrect rendering).
    Suggestion: Pipe the path through a jq substitution or sed to escape HTML entities, or switch Telegram to MarkdownV2 with proper escaping.

  3. Null Handling in Delta Computation (.github/scripts/generate_loc_report.sh:51-52)
    If loc_report.json.old exists but lacks the expected fields (e.g., corruption), jq returns null, causing bash arithmetic expansion to fail later.

    # Change to:
    OLD_TOTAL=$(jq '.total // 0' "$OLD_REPORT")
    OLD_CRATES_JSON=$(jq '.crates // []' "$OLD_REPORT")

Performance & Maintainability

  1. Compile-time Dependency (.github/workflows/daily_loc_report.yml:44)
    Compiling tokei from source (cargo install) adds ~2-3 minutes to every run. Use the pre-built binary instead:

    - run: |
        curl -L https://github.com/XAMPPRocky/tokei/releases/download/v12.1.2/tokei-x86_64-unknown-linux-gnu.tar.gz \
          | tar -xz -C $HOME/.cargo/bin
  2. Silent Curl (.github/scripts/publish_slack.sh:15, .github/scripts/publish_telegram.sh:25)
    Add -sS (silent with errors) to curl commands to prevent progress bars from cluttering the GitHub Actions log while still showing errors.

Minor Issues

  1. Arithmetic Expansion Safety (.github/scripts/generate_loc_report.sh:66-69)
    The format_diff function assumes inputs are valid integers. While the script sets defaults, explicit validation or defaulting to 0 on empty input would be safer.

  2. Cache Key Collision (.github/workflows/daily_loc_report.yml:69)
    The cache save key includes ${{ github.run_id }}, which is correct for the "always-save" pattern, but ensure the restore-keys prefix (loc-report-${{ github.ref_name }}-) is specific enough if you have multiple workflows writing similar cache keys.

Correctness Acknowledged

  • Proper use of set -euo pipefail in all bash scripts.
  • Correct handling of Slack/Telegram secret selection logic (test vs prod).
  • Appropriate use of actions/cache/restore and actions/cache/save split for immutable cache entries.
  • The count_loc function correctly excludes tests, benches, and examples.

Recommendation: Fix Item 1 (checkout version) and Item 3 (null handling) before merging.


Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt

@github-actions
Copy link
Copy Markdown

🤖 Claude Code Review

Here is the review of PR #368:


PR #368 Review: Daily LoC Report to Slack / Telegram

Overview: Adds a scheduled GitHub Actions workflow (daily_loc_report.yml) that counts Rust LoC per workspace crate using tokei + cargo metadata, computes day-over-day deltas via Actions cache, and posts formatted reports to Slack (daily) and Telegram (Monday only). Pure CI/infra change — no production Rust code is modified.


Bug: First-run failure on the Stash previous report as .old step

File: .github/workflows/daily_loc_report.yml, line ~54

- name: Stash previous report as .old for delta computation
  if: steps.cache-loc-report.outputs.cache-hit != ''
  run: mv loc_report.json loc_report.json.old

actions/cache/restore always emits cache-hit as the string 'true' or 'false' — never an empty string. So the condition != '' is always true. On the very first run of the workflow (no existing cache entry for this branch), cache-hit will be 'false', the step will run, mv will fail because no file was restored, and the job will exit non-zero.

Fix — use hashFiles to test actual file existence:

if: hashFiles('loc_report.json') != ''

Or change the run to be safe-by-default:

run: test -f loc_report.json && mv loc_report.json loc_report.json.old || true

actions/checkout@v6 likely does not exist

File: .github/workflows/daily_loc_report.yml, line ~37

As of this review, actions/checkout@v4 is the latest stable release. @v6 would silently resolve to the latest available semver-compatible tag, but if v6 doesn't exist it may fall back unexpectedly. Use actions/checkout@v4 to stay on a known-good version.


Telegram message length not bounded

File: .github/scripts/publish_telegram.sh + generate_loc_report.sh

The Telegram Bot API sendMessage endpoint caps messages at 4096 characters. With a growing number of crates, the HTML report could exceed this limit, causing the step to fail with a 400 Bad Request. There is currently no truncation or error handling for this.

A simple guard in publish_telegram.sh:

msg=$(cat "$MESSAGE_FILE")
if (( ${#msg} > 4096 )); then
    msg="${msg:0:4000}"$'\n\n<i>... (truncated)</i>'
fi

Or split the message into multiple API calls.


Suboptimal cat subprocess for Telegram body

File: .github/scripts/publish_telegram.sh, line ~26

--data-urlencode text="$(cat "$MESSAGE_FILE")"

curl natively supports reading from a file for --data-urlencode:

--data-urlencode text@"$MESSAGE_FILE"

This avoids a subshell, correctly handles binary-safe reads, and skips potential issues with large files hitting shell argument-length limits.


tokei exclusions only cover directory names, not inline test code

File: .github/scripts/generate_loc_report.sh, line ~19

tokei "$1" -t Rust --output json \
    -e tests -e benches -e examples 2>/dev/null

The -e flag excludes directories matching that name. It will not exclude inline #[cfg(test)] modules in .rs source files, which is the dominant test pattern in this codebase (unit tests are embedded next to the code they test). This is probably acceptable for a LoC trend report, but worth documenting as a known limitation so readers don't wonder why the numbers include test code.


Security: Slack webhook URL passed as a positional argument

File: .github/scripts/publish_slack.sh, line ~17

curl --fail-with-body -X POST "$WEBHOOK_URL" ...

The webhook URL is passed as $1 and becomes visible in the process list (/proc/<pid>/cmdline, ps aux) during the curl invocation. On GitHub Actions' ephemeral runners this is low-risk, but it would be marginally better to pass the URL via an env var and reference it that way:

# In the workflow
env:
  SLACK_WEBHOOK_URL: ${{ ... }}

# In the script, use ${SLACK_WEBHOOK_URL:?...} directly

Not blocking, just worth noting since the webhook URL grants posting access to the channel.


Minor: redundant empty-string guard in publish_slack.sh

File: .github/scripts/publish_slack.sh, lines 11–14

WEBHOOK_URL="${1:?webhook URL required}"

if [[ -z "$WEBHOOK_URL" ]]; then

${1:?...} only errors when the parameter is unset, not when it is set to an empty string. If an empty string is passed (e.g., publish_slack.sh ""), the positional check passes and the -z guard catches it. The logic is actually correct, but the comment ${1:?...} is misleading — it implies it guards against empty strings when it only guards against missing arguments. Consider replacing both with:

WEBHOOK_URL="${1:?webhook URL required}"
[[ -n "$WEBHOOK_URL" ]] || { echo "::error::Slack webhook URL is empty"; exit 1; }

What looks good

  • set -euo pipefail throughout all scripts — correct fail-fast behavior.
  • Secrets are accessed as env vars and never echoed; only the $SLACK_WEBHOOK argument exposure is a minor concern (above).
  • The caching strategy (run_id as exact key + branch-prefix restore-keys) is the correct pattern for a rolling "previous run" cache.
  • workflow_dispatch test/prod targeting and the post_telegram opt-in are clean UX for safe testing.
  • permissions: contents: read + actions: write is appropriately scoped — no write access to repo contents.
  • The format_diff_jq jq function handles all three delta cases (positive, negative, zero) correctly.

Summary: One definite first-run breakage (the mv step condition), one likely non-existent action version, and a few minor hardening points. None of the issues affect production Rust code. The first-run bug should be fixed before merging; the rest are improvements.


Automated review by Claude (Anthropic) · sonnet · custom prompt

@github-actions
Copy link
Copy Markdown

🤖 Codex Code Review

Findings

  1. Medium: daily_loc_report.yml and daily_loc_report.yml choose prod vs test destinations with cond && prod || test. In GitHub Actions expressions, that idiom falls through to the right-hand side when the selected prod secret/chat ID is empty. A scheduled run or target=prod manual run can therefore silently post to the test Slack channel / Telegram chat instead of failing on missing prod config. Use explicit if: branches or a shell step that selects exactly one destination and validates only that secret.

  2. Low: generate_loc_report.sh only scans each crate’s src/ directory. The report is described as workspace Rust LoC, but this misses crate-level Rust files like bin/ethlambda/build.rs. Totals and deltas will be wrong whenever build scripts or other non-src Rust targets change. Count the crate root and rely on the existing tests/, benches/, and examples/ exclusions instead.

No consensus-layer Rust paths are touched in this PR, so I didn’t see fork-choice/state-transition/XMSS/SSZ concerns beyond these workflow issues.


Automated review by OpenAI Codex · gpt-5.4 · custom prompt

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 13, 2026

Greptile Summary

This PR adds a new scheduled GitHub Actions workflow that counts Rust lines of code across all workspace members daily, posts a Slack Block Kit message every day, and sends an HTML Telegram message on Mondays (or on-demand). Day-over-day deltas are computed by caching and restoring the previous run's JSON report.

  • generate_loc_report.sh enumerates workspace crates via cargo metadata, counts LoC with tokei, and emits four output files (Slack JSON, Telegram text, GitHub step summary, and a cacheable JSON report).
  • publish_slack.sh and publish_telegram.sh handle delivery; routing between prod/test channels is done via a ternary expression in the workflow YAML environment block.
  • The workflow gates the Telegram step with a day-of-week check for scheduled runs and a boolean opt-in for manual dispatches.

Confidence Score: 4/5

Safe to merge; all changes are additive CI scripts with no impact on application code.

The core delta/cache logic and conditional routing are sound. The main actionable issue is in publish_slack.sh, where the :? expansion silently swallows the more descriptive check-your-secret error message that was clearly intended to guide operators on misconfiguration. Two other findings (webhook URL in argv, src/-only LoC scanning) do not break functionality.

publish_slack.sh needs the dead-code guard fixed so the informative error message is actually reachable.

Important Files Changed

Filename Overview
.github/scripts/generate_loc_report.sh Counts Rust LoC per workspace member via tokei + cargo metadata, computes day-over-day deltas from a cached old report, and writes Slack/Telegram/GitHub Step Summary payloads; LoC counting is silently skipped (0) for any crate lacking a src/ subdirectory.
.github/scripts/publish_slack.sh POSTs the Slack Block Kit payload via curl; the intended informative error message for a missing webhook secret is unreachable because the :? expansion already exits before the if [[ -z ... ]] guard.
.github/scripts/publish_telegram.sh Sends an HTML-formatted Telegram message; validation of required env vars is correct (uses :- to avoid unbound-var errors with set -u), and URL-encoding of the message body is handled by --data-urlencode.
.github/workflows/daily_loc_report.yml Defines the scheduled/manual workflow with correct cache-hit gating for delta computation, conditional Slack/Telegram routing, and a Monday-only guard for Telegram on scheduled runs; webhook URL is passed as a positional argument rather than an env var.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A([Trigger: schedule 0 0 * * * / workflow_dispatch]) --> B[Checkout + Setup Rust + tokei]
    B --> C[Restore cache\nloc-report-branch-runid]
    C -->|cache-hit != ''| D[mv loc_report.json to loc_report.json.old]
    C -->|cache-hit == ''| E[No delta baseline - first run]
    D --> F[generate_loc_report.sh]
    E --> F
    F --> G[Save loc_report.json to cache]
    G --> H[Append to GITHUB_STEP_SUMMARY]
    H --> I{schedule or target=prod?}
    I -->|yes| J[SLACK_WEBHOOK = GENERAL]
    I -->|no| K[SLACK_WEBHOOK = TEST]
    J --> L[publish_slack.sh]
    K --> L
    L --> M{schedule?}
    M -->|yes, day != Monday| N[Skip Telegram]
    M -->|yes, Monday| O[publish_telegram.sh]
    M -->|no / workflow_dispatch| P{post_telegram=true?}
    P -->|yes| O
    P -->|no| N
Loading
Prompt To Fix All With AI
Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 3
.github/scripts/publish_slack.sh:9-15
The `[[ -z "$WEBHOOK_URL" ]]` guard on lines 12-15 is dead code and can never be reached. The `${1:?webhook URL required}` expansion on line 9 already causes the script to exit with an error if `$1` is empty or unset — so by the time execution reaches the `if` block, `WEBHOOK_URL` is guaranteed to be non-empty. The informative `::error::` annotation with the secrets-troubleshooting message is therefore never surfaced to the operator.

```suggestion
WEBHOOK_URL="${1}"
PAYLOAD_FILE="${2:?payload file required}"

if [[ -z "$WEBHOOK_URL" ]]; then
    echo "::error::Slack webhook URL resolved to an empty value — check the secret configured for this trigger (scheduled vs manual)"
    exit 1
fi
```

### Issue 2 of 3
.github/workflows/daily_loc_report.yml:82
**Webhook URL passed as positional CLI argument**

`$SLACK_WEBHOOK` is expanded into the command line, making it visible in `ps` output for the duration of the `curl` call. Passing it via an environment variable (as is already done for `TELEGRAM_BOT_TOKEN`) keeps it out of the process argument list. The script already has `SLACK_WEBHOOK` as an env var, so you could read it directly from there instead of forwarding it as `$1`.

### Issue 3 of 3
.github/scripts/generate_loc_report.sh:42-47
**LoC count limited to `src/` subdirectory only**

Crates that place Rust source in directories other than `src/` — for example, a `proc-macro` sub-crate that uses a flat layout, or a virtual workspace member with code directly at the crate root — will be counted as 0 LoC. The tokei call would still find their `.rs` files if pointed at the crate root (`$dir`), but the current code skips any crate where `$dir/src` does not exist. This is worth a clarifying comment if the workspace layout is always conventional.

Reviews (1): Last reviewed commit: "ci: add daily LoC report workflow" | Re-trigger Greptile

Comment thread .github/scripts/publish_slack.sh Outdated
Comment on lines +9 to +15
WEBHOOK_URL="${1:?webhook URL required}"
PAYLOAD_FILE="${2:?payload file required}"

if [[ -z "$WEBHOOK_URL" ]]; then
echo "::error::Slack webhook URL resolved to an empty value — check the secret configured for this trigger (scheduled vs manual)"
exit 1
fi
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The [[ -z "$WEBHOOK_URL" ]] guard on lines 12-15 is dead code and can never be reached. The ${1:?webhook URL required} expansion on line 9 already causes the script to exit with an error if $1 is empty or unset — so by the time execution reaches the if block, WEBHOOK_URL is guaranteed to be non-empty. The informative ::error:: annotation with the secrets-troubleshooting message is therefore never surfaced to the operator.

Suggested change
WEBHOOK_URL="${1:?webhook URL required}"
PAYLOAD_FILE="${2:?payload file required}"
if [[ -z "$WEBHOOK_URL" ]]; then
echo "::error::Slack webhook URL resolved to an empty value — check the secret configured for this trigger (scheduled vs manual)"
exit 1
fi
WEBHOOK_URL="${1}"
PAYLOAD_FILE="${2:?payload file required}"
if [[ -z "$WEBHOOK_URL" ]]; then
echo "::error::Slack webhook URL resolved to an empty value — check the secret configured for this trigger (scheduled vs manual)"
exit 1
fi
Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/scripts/publish_slack.sh
Line: 9-15

Comment:
The `[[ -z "$WEBHOOK_URL" ]]` guard on lines 12-15 is dead code and can never be reached. The `${1:?webhook URL required}` expansion on line 9 already causes the script to exit with an error if `$1` is empty or unset — so by the time execution reaches the `if` block, `WEBHOOK_URL` is guaranteed to be non-empty. The informative `::error::` annotation with the secrets-troubleshooting message is therefore never surfaced to the operator.

```suggestion
WEBHOOK_URL="${1}"
PAYLOAD_FILE="${2:?payload file required}"

if [[ -z "$WEBHOOK_URL" ]]; then
    echo "::error::Slack webhook URL resolved to an empty value — check the secret configured for this trigger (scheduled vs manual)"
    exit 1
fi
```

How can I resolve this? If you propose a fix, please make it concise.

Comment thread .github/workflows/daily_loc_report.yml Outdated
${{ (github.event_name == 'schedule' || inputs.target == 'prod')
&& secrets.ETHLAMBDA_GENERAL_SLACK_WEBHOOK
|| secrets.ETHLAMBDA_TEST_SLACK_WEBHOOK }}
run: bash .github/scripts/publish_slack.sh "$SLACK_WEBHOOK" loc_report_slack.json
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Webhook URL passed as positional CLI argument

$SLACK_WEBHOOK is expanded into the command line, making it visible in ps output for the duration of the curl call. Passing it via an environment variable (as is already done for TELEGRAM_BOT_TOKEN) keeps it out of the process argument list. The script already has SLACK_WEBHOOK as an env var, so you could read it directly from there instead of forwarding it as $1.

Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/daily_loc_report.yml
Line: 82

Comment:
**Webhook URL passed as positional CLI argument**

`$SLACK_WEBHOOK` is expanded into the command line, making it visible in `ps` output for the duration of the `curl` call. Passing it via an environment variable (as is already done for `TELEGRAM_BOT_TOKEN`) keeps it out of the process argument list. The script already has `SLACK_WEBHOOK` as an env var, so you could read it directly from there instead of forwarding it as `$1`.

How can I resolve this? If you propose a fix, please make it concise.

Comment thread .github/scripts/generate_loc_report.sh Outdated
@dicethedev
Copy link
Copy Markdown
Contributor

@MegaRedHand please how can I stay connected more to development updates via telegram?

@MegaRedHand
Copy link
Copy Markdown
Collaborator Author

@dicethedev We give daily updates on Telegram, sometime post updates on X, and have weekly community calls on Fridays (we post the link on Telegram; it's live on https://x.com/class_lambda). The info is here in the CONTRIBUTING.md (we should probably add that on the readme and docs website 🤦).
For more general ecosystem updates, there's the open PQ Interop calls.

pablodeymo pushed a commit that referenced this pull request May 14, 2026
## Summary

- Add a **Community** section to the README and `docs/introduction.md`
so newcomers find the Telegram group, the weekly Friday community call,
and the PQ Interop calls without digging into `CONTRIBUTING.md`.
- Refresh the existing **Communication** section in `CONTRIBUTING.md` to
match (adds the weekly call and PQ Interop pointers, tightens the
Telegram description).

Triggered by the follow-up in
#368 (comment).

## Test plan

- [x] `README.md` renders the new Community section between Contributing
and Philosophy.
- [x] `docs/introduction.md` renders the new Community section before
Related projects (will show up in the mdBook).
- [x] `CONTRIBUTING.md` Communication block stays consistent with the
other two.
@dicethedev
Copy link
Copy Markdown
Contributor

@dicethedev We give daily updates on Telegram, sometime post updates on X, and have weekly community calls on Fridays (we post the link on Telegram; it's live on https://x.com/class_lambda). The info is here in the CONTRIBUTING.md (we should probably add that on the readme and docs website 🤦).
For more general ecosystem updates, there's the open PQ Interop calls.

Thanks for letting me know and I also saw your PR reference. LFG!

pablodeymo and others added 3 commits May 14, 2026 18:41
Replace the tokei + jq pipeline with cargo-warloc, which parses Rust
with `syn` and natively classifies inline `#[cfg(test)]` blocks as
test code. The report now shows per-crate counts (no tests) and two
totals at the bottom: with and without tests.

Reasons:
- tokei can only exclude directory names, so unit tests written
  inline (the dominant pattern here) leaked into the "production" LoC.
- Per-crate scanning of crate roots was also missing build.rs.

Implementation moves from bash+jq to a small Python script — Python 3
ships on ubuntu-latest, so no extra setup is needed beyond the
cargo-warloc install.
Passing the webhook URL as `$1` made it visible in the process list
(`ps aux`, `/proc/<pid>/cmdline`) for the duration of the curl call.
Switch to reading it from the `SLACK_WEBHOOK` env variable in the
script, matching the existing `publish_telegram.sh` pattern.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants